SPECIFICATION 



TO ALL WHOM IT MAY CONCERN : 

BE IT KNOWN THAT WE, Kouki Hayashi, a citizen of 
Japan residing at. Yokosuka-shi , Kanagawa Japan, Akira Hiraiwa, 
a citizen of Japan residing at Yokohama-shi , Kanagawa Japan, 
Hiroyuki Manabe, a citizen of Japan residing at Yokosuka-shi, 
Kanagawa Japan, Takashi Ninjouji, a citizen of Japan residing at 
Yokohama-shi, Kanagawa Japan and Toshiaki Sugimura , a citizen of 
Japan residing at Yokohama-shi, Kanagawa Japan have invented 
certain new and useful improvements in 



A ROBOT REMOTE MANIPULATION SYSTEM 
AND REMOTE MANIPULATION DEVICE 



Of which the following is a specification:- 
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TITLE OF THE INVENTION 

A ROBOT REMOTE MANIPULATION SYSTEM AND REMOTE 
MANIPULATION DEVICE 

5 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention generally relates to 
10 a robot remote manipulation system and a robot 

remote manipulation device, and specifically relates 
to such systems and devices that remotely manipulate 
bipedal walking robots via communication networks. 

2. Description of the Related Art 
A variety of robot remote manipulation 

systems have been developed recently. As one prior 
art robot remote manipulation system, Tmsuk 04 
(Robocon magazine No. 18, pp - 20-23, published by 
Ohm-sha in 2001) is known in which a robot can be 
remotely manipulated using a PHS line. In the Tmsuk 
04 technology, an operator operates a master arm to 
control the motion of a robot as a slave. Another 
prior art manipulation system uses a joy stick. By 
moving the joy stick forward, backward, rightward or 
leftward, the moving direction, the length of a step 
and the walking speed of a robot can be controlled. 

There is known one other prior bipedal 
30 walking robot remote manipulation system in which 
commands are inputted or a moving destination and 
route are inputted. 



20 



Such conventional bipedal walking robot 
35 remote manipulation systems, however, have 
comparatively large sizes. Further, while 
manipulating a robot in the conventional systems, it 



is not easy to be aware of forces applied to the 
legs of the robot and therefore it is not easy to 
control the robot. 

SUMMARY OF THE INVENTION 

Accordingly, it is one object of the 
present invention to provide a robot remote 
manipulation system and manipulation device, by 
which an operator can with ease remotely manipulate 
a bipedal walking robot and can be aware of forces 
applied to the legs of the bipedal walking robot. 

Another and more specific object of the 
present invention is to provide a robot remote 
manipulation system including a bipedal walking 
robot and a remote manipulation device for remotely 
manipulating the bipedal walking robot, the robot 
being connected to the remote manipulation device 
via a communication network and controlled by 
controlling data from the remote manipulation device, 

the remote manipulation device comprising: 

a pair of bilateral mechanical rotating 
elements each providing a quantity of motion for one 
of bilateral legs of the bipedal walking robot; and 

a controlling data transmitter for 
transmitting controlling data corresponding to the 
quantities of motion to the bipedal walking robot; 
and 

the bipedal walking robot comprising: 

a controlling data receiver for receiving 

the controlling data transmitted from the remote 

manipulation device; and 

a leg motion controller for processing the 

received controlling data and causing the bilateral 

legs to move forward or backward. 
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Another object of the invention is to 
provide a remote manipulation device for remotely 
manipulating a bipedal walking robot connected to 
5 the remote manipulation device via a communication 
network , comprising : 

a pair of bilateral mechanical rotating 
elements each providing a quantity of motion for one 
of bilateral legs of the bipedal walking robot; and 
10 a controlling data transmitter for 

transmitting controlling data corresponding to the 
quantities of motion to the bipedal walking robot. 

A further object of the invention is to 
15 provide a remote manipulating method in a robot 

remote manipulation system including a bipedal 

walking robot and a remote manipulation device for 

remotely manipulating the bipedal walking robot, the 

robot being connected to the remote manipulation 
20 device via a communication network and controlled by 

controlling data from the remote manipulation device, 

the method comprising the steps of: 

operating a pair of bilateral mechanical 

rotating elements in the remote manipulation device, 
25 and providing a quantity of motion for each of 

bilateral legs of the bipedal walking robot; and 
transmitting controlling data 

corresponding to the quantities of motion to the 

bipedal walking robot; 
30 in the bipedal walking robot, receiving 

the controlling data transmitted from the remote 

manipulation device; and 

processing the received controlling data 

and causing the bilateral legs to move forward or 
35 backward- 



Features and advantages of the present 
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invention are set forth in the description that 
follows, and in part will become apparent from the 
description and the accompanying drawings, or may be 
learned by practice of the invention according to 
5 the teachings provided in the description. Objects 
as well as other features and advantages of the 
present invention will be realized and attained by 
an apparatus particularly pointed out in the 
specification in such full, clear, concise, and 
10 exact terms as to enable a person having ordinary 
skill in the art to practice the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 FIG. 1 shows a block diagram of a robot 

remote manipulation system according to the present 
invention ; 

FIG. 2 is an exterior appearance view of 
the remote manipulation device shown in FIG. 1; 
20 FIG. 3 illustrates a controlling technique 

of a bipedal walking robot using the remote 
manipulation device shown in FIG. 1; 

FIG. 4 shows schematic views of a 
treadmill mechanism used in the remote manipulation 
25 device shown in FIG. 1; 

FIG. 5 illustrates a force applied to the 
bipedal walking robot on a slope; 

FIG. 6 is a schematic diagram of a circuit 
for feeding the force back to the treadmill; and 
30 FIG. 7 shows another example of feeding 

the force back to the treadmill; 

FIG. 8 shows a relationship between the 
quantity of motion of the treadmill and the step 
length of the bipedal walking robot; 
35 FIG. 9 is a flowchart for- illustrating a 

remote manipulation process of the robot using the 
remote manipulation system shown in FIG. 1; 
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FIG. 10 is a flowchart for illustrating a 
control process of the treadmill resistance using 
information obtained by an inclination sensor 
provided in the robot; and 

FIG. 11 shows schematic views of remote 
manipulation devices having rollers according to 
another embodiment of the invention. 



10 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following, embodiments of the 
present invention are described with reference to 
the accompanying drawings. 

15 

FIG. 1 shows a block diagram of a bipedal 
walking robot remote manipulation system according 
to a first embodiment of the present invention. 

20 The remote manipulation system shown in 

FIG. 1 includes a bipedal walking robot 20 (referred 
to as merely "robot" hereinafter) and a portable 
type remote manipulation device 10 capable of 
remotely manipulating the robot 20 via a 

25 communication network 100. 



The remote manipulation device 10 includes 
a CPU 19 for controlling all the elements, a 
treadmill 11 having bilateral rotational belt 
30 mechanisms,, a servo controller 12 for controlling 
outputs of motors for driving the rotational belts 
based on instructions from the CPU 19, and a D/A 13 
for D/A converting an input to the servo controller. 



35 



The remote manipulation device 10 further 
includes an encoder 14 for detecting rotational 
angles, speeds and directions of the motors, a 
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counter 15 for counting the number of pulses from 
the encoder 14, a display 16 for displaying 
audiovisual information sent from a CCD camera 
mounted on the robot, an operation part 17 such as a 
5 ten-key pad, and a radio communication part 18 for 
communicating with a base station (not shown) via 
the communication network 100. 

The robot 20 includes an inclination 
sensor 21 for sensing an inclination angle of the 
robot, a CPU 22 for calculating the status of the 
robot legs based on the inclination angle sensed by 
the inclination sensor 21, and a radio communication 
part 23 . The radio communication part 23 data- 
processes the status of the robot legs calculated by 
the CPU 22 , and transmits the processed data 
(referred to as *force sense data" hereinafter) to a 
base station (not shown) via the communication 
network 100. 

FIG . 2 is an exterior appearance view of 
the remote manipulation device 10 shown in FIG. 1. 

The remote manipulation device 10 in FIG. 
25 2 shows the display 16, the treadmill 11 and the 
operation part 17. As shown in FIG. 2, the 
treadmill 11 has a pair of bilateral rotational 
belts. According to the present invention, as shown 
in FIG. 3, the bilateral rotational belts can be 
30 rotated and moved by the index finger and the middle 
finger of an operator's dominant hand (right hand, 
for example) , and the right and left treadmills can 
receive their respective quantities of motion. The 
quantities of motion are used for manipulating the 
35 robot leg movements. 

The treadmill can be configured as shown 



15 
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in FIG. 4, for example. 

The treadmill 11 shown in FIG. 4 includes 
a rotational belt 31, a motor 32 for controlling 
5 resistance to the movement of the rotational belt 31, 
rollers 33 driven by the motor 32 to apply a 
resistance force to the rotational belt 31 wound 
around thereon, encoders 14 measuring the quantity 
of motion of the rotational belt 31 of the treadmill, 
10 and switches 35 sensing when the operator's finger 
touches the rotational belt 31. 

In this treadmill 11, the belt can be 
rotated only when the switch 35 is ON. The encoders 

15 14 (incremental encoders, for example) output pulses 
corresponding to the rotational angles of the 
rollers 33. By counting the pulses outputted from 
the encoders, the quantity of motion of the 
treadmill 11 can be determined. Instead of the 

20 incremental encoder, an absolute encoder can be used, 
of course. In the case of the absolute encoder, 
outputted absolute rotational angle is deemed to 
represent the treadmill motion quantity. 

25 In this embodiment, when the robot walks 

up or down on a slope, the resistance to the 
movement of the rotational belt 31 of the treadmill 
11 is controlled and made heavy or light, 
respectively. In more detail, the inclination 

30 sensor 21 mounted on the robot 20 measures the 

status of the robot 20. When it is determined that 
the robot is going up a slope based on the data 
measured by the inclination sensor 21, the belt 
movement of the treadmill 11 is made heavy. When it 

35 is determined that the robot is going down a slope 

based on the data measured by the inclination sensor 
21, the belt movement of the treadmill 11 is made 
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light- This resistance control is explained below. 

FIG . 5 illustrates a force applied to the 
bipedal walking robot 20 walking on a slope. 

5 

In FIG. 5, when the robot 20 is going to 
walk up the slope having an inclination angle 9 , a 
force Mgsin 9 is applied to the robot in the down- 
slope direction, where M represents the mass of the 

10 robot and g represents the acceleration of gravity. 
In this case, a force C 0 Mgsin 0 generated by the 
motor 32 is applied to the treadmill 11 (Co is a 
constant) , and therefore the force applied to the 
robot 20 is indirectly fed back to the operator's 

15 fingers on the treadmill 11. 

When the upslope is steeper ( 6 becomes 
larger) , the motor 32 provides a feed-back force to 
the treadmill 11 and makes it difficult to move the 

20 belt 31 of the treadmill 11. On the other hand, 
when the slope is downward ( 0 becomes negative) , 
the motor 32 provides a feed-back force to the 
treadmill 11 and makes it easy to move the belt 31 
of the treadmill 11. These feed-back forces are 

25 generated only when the operator's finger touches 
the belt 31 of the treadmill 11 (only when the 
switch 35 mounted on the treadmill 11 is ON) . 

In this embodiment an electric circuit as 
30 shown in FIG. 6 is used for generating the above 
force C o Mgsin0 against the treadmill 11. 

FIG. 6 is a schematic diagram of an 
electric circuit for feeding the above force back to 
35 the treadmill 11. The circuit includes a motor 32 

generating a torque T, a battery having a voltage V, 
a switch 35 and a variable resistance Ra 40. The 
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variable resistance Ra can be varied depending on 
force sense data transmitted from the robot, and 
therefore a voltage applied to the motor 32 can be 
varied according thereto. As a result, the torque T 
5 can be controlled so that the motor 32 generates a 
force CoMgsin© corresponding to the slope 
inclination 0 , and therefore the resistance of the 
belt 31 of the treadmill 11 reflects the status of 
the robot walking on the slope having the 
10 inclination angle 0 . 

The resistance Ra in the above electric 
circuit can be obtained in the following equation 
(1) - 

15 



20 



Ra = —(V-K*a>) (1) 
T 



This Ra can be represented as in the 
following equation (2) , where the quantity of motion 
for the slope 6 is Al 



Ra = L-—<y-CsW (2) 

C 0 »M •g»sm6 



Where K is a counter-electromotive force 
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constant of the motor; Al is the quantity of motion 
of the treadmill; and Co, Ci are constants. 

FIG. 7 shows another example of feeding 
5 the force back to the treadmill 11. 

As shown in FIG. 7, this example uses a 
bar-like module 45, which is pressed against the 
treadmill 11 by a force F to control the resistance 

10 of the belt 31 of the treadmill 11. Where the 

friction coefficient between the belt 31 and the 
module 45 is represented by M , the force F 
generates a friction force /i F on the belt 31. 
Accordingly, by varying F so that V F becomes equal 

15 to CoMgsin6 , the treadmill 11 can receive a 
backward force of C o Mgsin0. 

The manner for feeding a force back to the 
treadmill is not limited to the above explained 
20 methods and can be fed back in a variety of ways 
within the scope of the present invention. 

Next f referring to FIG. 8, the 
relationship between finger motion quantity on the 

25 treadmill 11 and robot leg movement is explained. 
If a quantity of motion of the left treadmill 
(operated by an index finger) is represented by Al x 
and a quantity of motion of the right treadmill 
(operated by a middle finger) is represented by Al r , 

30 movements of left and right leg steps are 

represented by Al l and Al r , respectively, then the 
relation as Al) : A l r = A L L : Al r holds. The 
distance between two legs of the robot is 
represented by D , 

35 

Where the minimum turning radius of the 
robot 20 is represented by R, the curvature f> 
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having this radius R is obtained by the following 
equation (3 ) : 



Where the quantity of motion of the left 
treadmill is smaller, the robot turns to the left. 
When the quantities of motion of the left and right 
treadmills are equal, the robot walks straight. On 
the other hand, by moving the treadmill backward the 
robot can walk backward. Therefore, it is possible 
to manipulate the bipedal walking robot freely, such 
as walking forward, walking backward and turning. 

Referring to a flowchart shown in FIG. 9, 
a process for manipulating the robot legs using the 
remote manipulation system in FIG. 1 is explained. 

In FIG. 9, the robot 20 and the remote 
manipulation device 10 communicate with each other 
via a communication network to exchange information. 



treadmill 11 of the remote manipulation device 20 
(51) , the encoder 14 measures the number of pulses 
outputted by the treadmill 11 . The number of pulses 
corresponds to an angular amount of movement of the 
rotated belt. The measured result is transmitted to 
the CPU 19. The CPU 19 calculates a quantity of 
motion of the treadmill 11, based on the measured 
results sent by the encoder 14 (S2). The quantity of 
motion of the treadmill 11 is converted to step 





When an operator's finger moves on the 
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length data of the robot 20 (S3), and the converted 
step length data are transmitted to the radio 
communication part 18. The radio communication part 
19 performs a coding process, a modulation process, 
5 and a frequency conversion process and other 

processes on the step length data, and transmits the 
processed data to a base station (not shown) via the 
communication network 100 (S4) . 

10 The step length data transmitted by the 

remote manipulation device 10 in the above mentioned 
manner is received at the radio communication part 
23 of the robot 20 via the communication network 100. 
After receiving the step length data transmitted by 

15 the remote manipulation device 10, the radio 

communication part 23 of the robot 20 performs a 
frequency conversion process, a demodulation process, 
a decoding process and other processes on the step 
length data, and outputs the processed step length 

20 data to a mechanism for controlling the legs of the 
robot. Then the legs of the robot 20 are controlled 
based on the processed step length data (S5) . 

In the above explained embodiment, an 
25 operator of the remote manipulation device 10 can 

remotely manipulate the robot 20 while monitoring an 
image on the display 16, which is sent from a CCD 
camera (not shown) mounted on the robot 20- However, 
if the operator can directly see the robot 20 at a 
30 near distance, he/she does not have to monitor the 
display 16 for manipulating the robot 20. 

In the above embodiment, the conversion 
calculation from the treadmill motion quantity to 
35 the robot step length is carried out by the CPU 19 
in the remote manipulation device 10. The present 
invention is not limited to such embodiment. For 
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example, the remote manipulation device 10 may 
perform the transmission of the treadmill motion 
quantity only, and CPU 22 in the robot 20 can then 
perform calculations relating to conversion from the 
5 treadmill motion quantity to the robot step length. 

Next, referring to the flowchart shown in 
FIG. 10, a process for controlling the treadmill 
resistance based on information obtained by the 
10 inclination sensor 21 mounted in the robot 20 is 
explained . 

The inclination sensor 21 mounted in the 
robot 20 senses an inclination angle of the robot 20 

15 (Sll), and sends the inclination information 

indicating the status of the robot legs to the CPU 
22. After receiving the inclination information, 
the CPU 22 calculates the mechanical status of the 
robot (whether the robot is on an up slope or a down 

20 slope) based on the inclination information (S12) , 
and calculates forces applied to the legs of the 
robot (S13) : 

The calculated results are sent from the 
25 CPU 22 to the radio communication part 23. The 
radio communication part 23 performs a coding 
process, a modulation process, a frequency 
conversion process and other processes on the 
calculated results, and transmits the converted 
30 results as force sense data to a base station (not 
shown) via the communication network 100 (S14). 

As explained above,, the force sense data 
transmitted by the robot 20 is received at the radio 
35 communication part 18 of the remote manipulation 

device 10. The radio communication part 18 performs 
a frequency conversion process, a demodulation 
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process , and a decoding process on the force sense 
data, and outputs the processed data to the motor 32 
of the treadmill 11 of the CPU 19 (S15) . The motor 
32 of the treadmill 11 outputs a certain force based 
5 on instruction from the CPU 19. Accordingly, the 
belt 31 of the treadmill 11 rotates with resistance 
corresponding to the force applied to the robot legs, 
and therefore the operator can manipulate the steps 
of the robot 20 while being aware of the walking 
10 condition of the robot 20. 

In the above embodiment, the calculation 
of the force applied to the robot legs is carried 
out by the CPU 22 in the robot 20. The present 

15 invention is not limited to such embodiment. For 
example, the robot 20 may perform the transmission 
of the output of the inclination sensor only, and 
the CPU 19 in the remote manipulation device 10 can 
then calculate the force applied to the legs of the 

20 robot 20 on a slope based on the output from the 
inclination sensor. 



Referring to FIG. 11, a second embodiment 
of the present invention is explained. According to 

25 the second embodiment, a remote manipulation device 
10 includes a pair of bilateral rollers 50 as shown 
in FIG. 11, instead of the treadmill 11. Using the 
rollers, the bipedal walking robot can be remotely 
manipulated. The rollers shown in the left device 

30 in FIG. 11 are spaced apart from each other and can 
be operated with thumbs of both hands. The rollers 
shown in the right device in FIG. 11 are close 
together and can be operated with one or two fingers 
on one hand. 

35 

The remote manipulation device may have 
other types of mechanisms such as a pair of gear 



mechanisms used in portable audio equipment. 



The above explained remote manipul ation 
device 10 may be any kind of mobile terminal 
connectable to a communication network such as 
mobile radio terminals (e.g. mobile phones) 
connectable to a mobile communication network, 
notebook computers, and PDAs. The communication 
network can be any kind of network such as a public 
network, a radio LAN , and an IP network, as long as 
the mobile terminals have a radio communications 
interface that can be used to connect to the 
utilized communication network- 



According to the embodiments of the 
present invention, the operator of the remote 
manipulation device can remotely manipulate robot 
steps by moving his fingers in a manner of 
simulating human walking while being aware of 
mechanical conditions such as forces applied to the 
robot, which awareness cannot be obtained through a 
display. And the remote manipulation device can be 
miniaturized . 

Further, real time manipulation can be 
attained because the system utilizes a mobile 
communication network. The walking area of a 
manipulated robot is not limited, because the robot 
can utilize a mobile communication network and its 
roaming services . 

The present application is based on 
Japanese Priority Patent Application No. 2002-241075 
filed on 8/21/2002 with the Japanese Patent Office, 
the entire contents of which are hereby incorporated 
by reference. 



